STABILITY OF TAILS AND 4-CANONICAL MODELS 
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Abstract. We show that the GIT quotients of suitable loci in the Hilbcrt 
and Chow schemes of 4-canonically embedded curves of genus g > 3 are the 
moduli space M^ B of pseudostable curves constructed by Schubert in [9] using 
Chow varieties and 3-canonical models. The only new ingredient needed in the 
Hilbert scheme variant is a more careful analysis of the stability with respect 
to a certain 1-ps A of the m th Hilbert points of curves X with elliptic tails. We 
compute the exact weight with which A acts, and not just the leading term in 
m of this weight. A similar analysis of stability of curves with rational cuspidal 
tails allows us to determine the stable and semistable 4-canonical Chow loci. 
Although here the geometry of the quotient is more complicated because there 
are strictly semistable orbits, we are able to again identify it as M p g s . Our 
computations yield, as byproducts, examples of both m-Hilbert unstable and 
m-Hilbert stable X that are Chow strictly semistable. 



Introduction. Over 30 years ago, Gieseker [1,2] gave a GIT construction of the 
moduli space M g of stable curves X of genus g as a quotient of a suitable locus 
in the Hilbert scheme of i/-canonically embedded curves and Mumford [8] gave a 
variant using Chow points. A condensed exposition is given in [3, Chapter 4]. These 
constructions require that v > 5. Since w® 1 ' is very ample when v > 3, a natural 
question is to describe the quotient when v equals 3 or 4. 

The only point in Gieseker 's construction at which the hypothesis v > 5 can- 
not be weakened is in applying the calculation that ordinary cusps (singularities 
locally analytically isomorphic to y 2 — x 3 ) destabilize Hilbcrt and Chow points of 
curves of degree d in P™ -1 when - < i: for i/-canonical curves - = „ 2v , . For a 

° n 7 n 2v—l 

proof with this constant, which is sharp as we will sec in Lemma 5, see [8, Propo- 
sition 3.1]; most references — [1, Proposition 1.0.5], [2, Lemma 5.7] and [3, Step 
4 on p. 251] — give the weaker bound | which is easier to verify and suffices for 
Giesckcr's application. In view of this result, v > 5 is needed implicitly in the proof, 
by semistable replacement, that all reducible stable curves are Hilbert stable. 

Recall that an elliptic tail is a connected genus 1 subcurve of a stable curve 
meeting the rest of the curve at a single point and a varying elliptic tail is a curve 
P 1 C M g obtained by gluing an elliptic tail of varying j-invariant to a fixed point on 
a fixed curve of genus g—1. Recall also that a divisor on M g is said to have slope s if 
it is a positive multiple of sA — S. On the one hand (cf.[3, §3.F]), divisors of slope 11 
are numerically trivial on varying elliptic tails. On the other, Mumford [8, Section 5] 
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(see also [4,7]) showed that, exactly when v < 4, the polarizations with which GIT 
naturally equips ^-canonical quotients have slope s < 11. 

These observations suggest that ordinary cusps appear and elliptic tails disappear 
in the ^-canonical quotients for v < 5. This prediction was verified by Schubert [7,9] 
who, using 3-canonical Chow points, produced, for g > 3, a quotient that is a 
coarse moduli space M P g for pseudostable curves: these are reduced, connected, 
complete curves with finite automorphism group, whose only singularities are nodes 
and ordinary cusps, and which have no elliptic tails. If g = 2, the subcurve C 
in [9, Lemma 4.1] is not unique and the moduli problem is not separated but 
this quotient was later constructed by Hyeon and Lee [5] who showed that all 
pseudostable cuspidal curves are identified in it. Recently Hassett and Hyeon [4] 
have analyzed the case v — 2 producing moduli spaces in which cusps and tacnodcs 
are allowed but elliptic tails and certain elliptic chains are excluded. But the case 
v = 4, whose GIT you'd expect to be easier, has never been treated. The aim of 
this short note is to fill in this gap and confirm the expectation that the quotient 
is again M^ s . We work throughout over an algebraically closed field k of arbitrary 
characteristic. 

A careful reading of Schubert's paper shows that, with one exception, all the 
instability and stability claims made in Sections 1-3 for v — 3 remain valid when 
v — 4 with only minor numerical modifications to reflect the change in v. As we 
will see in Corollary 2, his proof that curves with elliptic tails have Chow unstable 
3-canonical models only implies that their 4-canonical models arc not Chow stable. 

Schubert's argument can be rescued in two ways. The first is to work with Hilbert 
points. Indeed, his construction applies, mutatis mutandis, using Hilbert points of 
3-canonical curves; there are no Chow strictly semistable points and Chow stability 
or instability implies the corresponding Hilbert property (cf. [4, Proposition 3.13]). 
To make the argument apply to Hilbert points of 4-canonical curves requires only a 
more precise analysis of the weights with which an explicit one-parameter subgroup 
(henceforth 1-ps) p acts on the m th Hilbert points of certain projective embeddings 
of stable curves with elliptic tails. It would suffice to bound below the quadratic 
and linear terms in m of these weights, but, we are able to compute them exactly 
in Lemma 1. This calculation leads easily to the conclusion in Corollary 2 that 
771 th Hilbert points of 4-canonical models of stable curves with elliptic tails are 
p-unstable. 

Substituting Corollary 2 for his Lemma 3.1, the remainder of Schubert's con- 
struction, with the Chow scheme for 3-canonical curves replaced by the m th Hilbert 
scheme of 4-canonical curves for a sufficiently large m, goes through with only the 
sort of numerical changes mentioned. 

This leaves open the question of the 4-canonical Chow quotient. Here the quo- 
tient again turns out to be M g but the geometry is a bit more complicated. The 
closures of the Chow orbits both of cuspidal curves and of curves with elliptic tails 
contain the orbits of curves with (rational) cuspidal tails and all three types are 
Chow strictly semistable. To see that none are stable, we first compute the weight 
with which the same A acts on 4-canonical curves with cuspidal tails by an even 
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more direct argument in Lemma 3. This allows us to compute basins of attraction 
for this A and its inverse in Lemma 4. A semistable replacement argument (Propo- 
sition 6) then shows that these orbits are not unstable. This lets us, in Theorem 7, 
use the cycle map to see that the Chow and Hilbert quotients are isomorphic and 
hence identify the former with M g . 

Corollary 2 also shows that the 4-canonical Chow points of curves X with elliptic 
tails are strictly semistable and Hilbert unstable with respect to p and Lemma 3 
gives the same conclusions for Y with cuspidal tails. Our 4-canonical Chow quotient 
construction shows that such curves are Chow strictly semistable. Likewise, as a 
complement to Lemma 4, we compute in Lemma 5 the weights with which p^ 1 
acts on a 4-canonical cuspidal curve Z and show that such curves are Chow strictly 
semistable and m-Hilbert stable with respect to p^ 1 . The same conclusions follow 
unconditionally (i.e. for any 1-ps) for such Z from the constructions of the 4- 
canonical Chow and Hilbert quotients. 

Such X and Y are the easiest examples known to us of projective embeddings 
with strictly semistable Chow point and unstable m-Hilbert points and such Z the 
only ones with strictly semistable Chow point and stable m-Hilbert points. Both 
sets of examples are, in a sense, optimal. Corollary 2 and Lemma 3 show that, with 
respect to a critical 1-ps, their Hilbert-Mumford indices are, respectively, — (m— 1) 
and (m — 1) and [4, Proposition 3.17] implies that these indices must be divisible by 
(m — l). 1 Other examples of the former type arise in the analysis of bicanonically 
embedded rosaries in [4, Propositions 10.1 and 10.7]. Any Veronese embedding of 
a projective space is both Chow and Hilbert strictly semistable with respect to any 
p lying in its stabilizer, and also unconditionally. 
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University Faculty Fellowship and the paper was written while he was visiting the 
University of Sydney; he thanks both institutions and Gus Lehrer, Amnon Neeman 
and Paul Norbury for their hospitality during his time in Australia. 

The 1-ps p and stability with respect to it. If X is a Deligne-Mumford stable 
curve that can be written X = C U E where C and E are subcurves of genera 
(g — 1) and 1 respectively and C n E is a single node p, then we say that X has an 
elliptic tail. We do not assume C smooth or irreducible. Let I be a very ample, 
non-special line bundle on X of degree d whose restriction to E is linearly equivalent 
to E (vp) and let n = d — g + 1 = H°(X,L) and c = d — v = deg c (L). In our 
applications, v = 4 and L = , but for the moment using general d, n and v 
simplifies notation. 

Consider the embedding of X in P n_1 associated to L. It follows directly from 
Riemann-Roch that the linear spans Vq of C and V E of E in P™" 1 are of dimensions 



This divisibility is used in proving Lemma 3, but it gives a check on the calculations of 
Corollary 2 and Lemma 5. 
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c — g + 1 = n— v and v — 1 respectively and that their intersection is {p} . Letting 
I = n — v + 1 , we can therefore choose homogeneous coordinates x 1 , . . . , x n such 



that Xi 



x l _ l — defines V El x 



i+i 



defines V c , and p is the 



point where all the x i except x t vanish. 

For j > 1, we will confound Xi + j with the section of H°(E, L\ e ) it determines 
and write ord p (xi + j) for the order of vanishing at p of this section. Again by 
Riemann-Roch, we may choose x i+ j so that ord p (x ;+J ) = j for 1 < j < v — 2 and 
choose ■Z* so that ord p (a;„) = 

Define p to be the 1-ps subgroup of SL(n) acting by diag(i ri , • • • ,i r ") in these 
coordinates where 

v if i < I 
r, = < v — j if i = I + j and 1 < j < v — 2 







iii = n= l + i> — 1 



Note that, for j > 0, this gives weight equal to e — ord p (xi + j). 

Our sign convention is that the Hilbert-Mumford index /i([X] m , p) is minus the 
least weight with which p acts on an m th -Hilbert point of (X, L) with respect to 
the canonical linearization of the SL(n)-action. So our Numerical Criterion is 



(1) 



K[ x ]m,P) = -(w p {m) - mP(m)a(p)) 



where w p (m) is the least weight of any basis of H°(X,L® m ) consisting of the re- 
strictions to X of monomials of degree m in the a(p) is the average p- weight of 
a coordinate on P™ -1 and P(m) is the Hilbcrt polynomial of (X,L). Thus, with 
respect to p, [X] m is Hilbert stable, strictly semistable or unstable if and only if 
(w p (m) — mP(m)a(p)) is, respectively, negative, or positive and hence p([X] m , p) 
has the opposite sign. Both terms are numerical (that is, integer valued) polynomi- 
als of degree 2 in m and Chow stability for (X, L) is similarly determined by the sign 
of the leading coefficient of the difference. For details, see, for example, [3, Propo- 
sition 4.23 and Lemma 4.26] or [7, Proposition 3]). 

In our examples, P(m) = md — g + 1 by Riemann-Roch and an easy calculation 
shows that 

i=i 3-v v 2 -v + 2 

a(p) = J - = v - 



2n 



so that 



(2) mP(m)a(p) = nt 



d(v- 



2n 



2n 



Stability of elliptic tails. The proof of Schubert's Lemma 3.1 applied to our 
setup, shows that the m 2 coefficient of w p (m) is at least {d — (although only 
L = cj^ 3 is considered). The next lemma is the sharpening of this estimate to an 
exact evaluation of w p {m) needed to apply his argument to 4-canonical models. 



Lemma 1. For m > 2. w p (m) = m 2 



1. 
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Proof. For concision, wc will henceforth understand all monomials to have degree 
m and view them directly as sections of L® m over X or E (eliding "the restriction 
to"). Weights will always be p- weights. 

Let W r be the span in H°(X, I 8 ™) of all monomials of weight at most r and 
let s = mv — r. We claim that 

md — g + 1 if r = mv 
dim(W r ) = { r if 2 < r < mv - 1 

1 if r = or r = 1 

Given this, the lemma follows by elementary manipulations since the weight of any 
monomial basis of minimal weight is simply the sum of rf dim(W r ) — dim(W r _ 1 )) 
over r. 

The first case in the claim is immediate from Riemann-Roch for L® m on X. The 
others follow from the equality 

(3) W r = H°(E, L® m \ B (-sp)) for r = and for 2 < r < mv - 1 . 

Since L® m \ E (— sp) = E (rp) — recall that L\ E = E (vp), Riemann-Roch on E 
implies that h°(E, L® m \ E (-sp)) = r. 

If any monomial has weight r = mv — s then it contains one or more factors x i+ j 
with j > and hence vanishes on C . By construction, s equals the sum of the orders 
of vanishing at p of the factors of this type, hence W r C H a (E, L® m \ E {— sp)). 

If we next set M = x™, then B := {M } is a basis of H°(E, L® m \ E (-mvp)) 
lying in W . Finally, for r = 2, . . . , mv — 1, let M r be any monomial 
m tci 
M r := J^J x i+j k s -t- each j k > and j k = s = mv — r . 

k=l k=l 

Then, M r vanishes on C because some j k > 0. By construction, M r has weight 
exactly r and, since is non-zero at p, M r vanishes to order exactly s at p. Thus, 
B r := {M , M 2 ,M 3 , M r } is a subset of W r nH°(E, L® m \ E (-spj) of cardinality 
equal to h°(E, L® m \ E (—sp)) . But all the elements of B r except M r lie in W r _ 1 . By 
induction, B r is linearly independent and hence is a basis of H° [E, L® m | B (— sp)) 
which therefore lies in W r . □ 

We now specialize to the case where v = 4 and L = cj^ 4 so that d = 8(g — 1) and 
n = 7(g — 1). From Lemma 1 we obtain w p (m) — (32g — 40)m 2 + (— 4g + 6)m — 1 
and, from (2), mP(m)a(p) = (32g - 40)m 2 + (-4g + 5)m. 

Corollary 2. If X is a A-canonically embedded Deligne-Mumford stable curve with 
an elliptic tail then p([X] ml p) = — m + 1. Hence, with respect to p, such X are 
Chow strictly semistable and m-Hilbert unstable for any m > 2. 

There are many other examples with the last properties. For any v we can take 
a = -^2, d = a(g— 1) and n = (a— 1)(<?— 1) for g — 1 divisible by v — 2. When v = 3, 
g can be arbitrary. We have checked that none of these Chow strictly semistable 
examples are Hilbert semistable. 

Lemma 1 also reproves Schubert's Lemma 3.1 by taking v = 3 and L = w® 3 . 
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Stability of cuspidal tails. In this section, we consider only 4-canonical curves 
of genus g (so d = 8(g — 1), n — 7(g — 1) and I = 7g — 10). We want to consider a 
curve Y with a cuspidal tail: that is, Y is the join of a curve C of genus g—1 and a 
rational cuspidal curve R (of arithmetic germs 1) meeting C in a single node at p. 

The key remark is that the definition of p extends unchanged to these curves. 
Once again lo y \r — Or(p), we may choose coordinates such that X\ = . . . = = 
defines V Rl x i+1 = . . . = x n = defines Vq, and p is the point where all the x i 
except Xi vanish. Moreover, if we parameterize R by 



then ordp = j for j > 1 with its p-weight is the power n — I + j of t. Finally, 
p stabilizes R, and, as in Corollary 2: 

Lemma 3. IfY is a 4- canonical curve with a cuspidal tail, then p([Y] m , p) = m—1. 
Hence, with respect to p, such Y are Chow strictly semistable and m-Hilbert unstable 
for all m > 2. 

Proof. The approach of Lemma 1 works here but there's a more elementary and con- 
crete alternative. Because 4-canonical curves are embedded by complete linear se- 
ries with no higher cohomology as 2-regular subschemes in the sense of Castelnuovo- 
Mumford, we can apply Proposition 3.17 of [4]. It asserts that Wp(m) — (m — 
l)(am + b) for all m > 2 and hence is determined by Wp(2) and w F (3) (by an 
explicit formula that we will not need). 

The degree two monomials not in the initial ideal m p (R) of R must pullback to 
s 8 ~V (of weight i) where i runs from to 8 skipping 1. Hence their weights sum 
to 35. On the other hand, p acts on D with constant weight 4 and the number of 
monomials not in the initial ideal of D vanishing at the point of attachment p is 
h°(L® 2 (-p)) = 15g - 23 where L = wg 4 (4p). Hence the total weight in degree 2 is 
2 • 4 • (15g — 23) + 35 = 120g — 149. Likewise, the degree three monomials not in 
m p (R) pull back to s 12 ~ l t l with < i < 12 and i ^ 1 and have weights summing 
to 77. Adding the weight 3 • 4 • h?{D,L m {-p)) = 12(23.g - 35) of any monomial 
basis of sections on D vanishing at p, we get a total weight in degree 3 of 276g — 343. 

On the other hand, (2) gives mP(m)a p — m(8m — l)(4g — 5) which yields 
120#™150 for to = 2 and 276#-345 for to = 3. This shows that p([Y] m ,p) = -m+1 
for to = 2 and 3 and hence for all to. □ 

Let G be a reductive group, H C P(V) be a projective variety with a linear G 
action. Let p : G m — *-G be a one parameter subgroup with fixed points H p . For 
each x* € H p , the p-basin of attraction is defined 



If a; € A p (x*) and p(x* , p) — 0, then x is semistable if and only if x* is. Since the 
orbit closure of any x S A p {x*) contains x* , if p(x*,p) = and x* is semistable, 
then x and x* are identified in the quotient space H//G. We want to apply this to 
the Chow point [Y] of the curve with a cuspidal tail. 



[s,t]i — ^0, . . . ,0,t 4 , st 3 , s 2 t 2 , s% 
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Figure 1 . Basin of attraction of Y 

Lemma 4. The basin of attraction AMY]) [resp. A p -i([Y])] contains smoothings 
of the cusp [resp. node] but no smoothings of the node [resp. cusp]. 

Proof. At the cusp q — [0, . . . , 0, 1], the deformation space is given by y 2 — x 3 + 
ax + b, and p acts on x (which may be taken to be (the image of) "~ 1 in the 
completed local ring) with weight 2. Hence p acts on a and b with weights 4 and 
6, and the basin of attraction contains smoothings of the cusp. On the other hand, 
local coordinates at the node are given by dividing by x n _ 3 . In these, p acts on 
the tangent line to E at the node p (spanned by ^"~ 2 ) with weight —1 and on the 
tangent line to C at p trivially. Hence the basin of attraction does not admit a 
smoothing of the node. Since p~ x acts on the deformation spaces with weights of 
the opposite signs, the opposite conclusions hold. □ 

This reproves Corollary 2. As a complement, we sketch a direct proof that 
Z is m-Hilbert stable and Chow strictly semistable with respect to p~ l which we 
normalize to have weights [0, . . . , 0, 1, 2, 4] with Xi = . . . = = for k = n—l,n—2 
and n — 3 defining the cusp q, its tangent line and its osculating plane respectively. 
Thus x n , x n _x, x n _ 2 and x n _ 3 vanish, respectively, to orders 0, 1,2 and 4 at q. 

Lemma 5. If Z is a ^-canonical curve with a cusp, then p([Z] m , p^ 1 ) = m— 1. 
Hence, with respect to p , such Z are Chow strictly semistable and m-Hilbert stable 
for all m> 2. 

Proof. For j > 0, x l+ j has p^ 1 - weight equal to 4 — ord q (x[ + j). This implies, in the 
notation of Lemma 1 (for p- 1 ), that W 4m _ s = H n (Z,u® im {-sq)) for < s < 4m. 
Thus, dim(W r /W r ^ 1 ) = 1 for < r < 4m - 2 and dim(H / 4 m /T4 / 4 m _ 2 ) = 1 and 
the least weight of a basis is (X)r=i r ) — — -0 = — ^ m + 1- Since the 
weights total 7, mP(m)a p -i = m(8m — l)(g — 1) 7 ( g !_i) = $m 2 — m, which gives 
the claimed p. □ 
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Proposition 6. The ^-canonical Chow points of X, Y and Z are Chow strictly 
semistable. 

Proof. In view of Corollary 2 and Lemmas 3 and 5, we must eliminate the possibility 
that Y is Chow unstable. By Lemma 4, we can do this by showing that any of the 
three curves is Chow semistable. 

Let B — Spec(fc[[i]]) with generic point r\ and closed point and let S — *~B be 
a family of curves over B with generic fiber smooth of genus g and special fiber 
S isomorphic to Y. After making a base change, if necessary, we can assume that 
there is a family S' — *-B with general fiber S' n isomorphic to S v and with Chow 
semistable special fiber S' . 

Schubert's arguments [9, Section 2] show that S' can have only nodes and cusps 
as singularities and that it cannot contain any semistable chains of smooth rational 
curves. By stable reduction, on the other hand, S' must have the same Delignc- 
Mumford stabilization as Y. These two conditions force S' to be one of the curves 
X, Y or Z. □ 

The 4-canonical GIT quotient spaces. Let Hilb g iy denote the closure of the 
locus of smooth ^-canonically embedded curves in the Hilbcrt scheme of curves 
of genus g and degree 2v(g — 1) in pi 2 " -1 ^ -1 )- 1 and let Chow ff „ denote the 
corresponding locus in the Chow variety. A complete connected curve is weakly 
pseudostable if it has only nodes and ordinary cusps as singularities and any smooth 
rational component meets the rest of the curve in at least three points. 

Combining Schubert's arguments with Corollary 2, Lemmas 3 and 5, and Propo- 
sition 6 gives: 

(1) In Hilb g 4 , the stable locus parameterizes pseudostable curves, and there 
are no strictly semistable points. 

(2) In Chow g 4 the semistable locus parameterizes weakly pseudostable curves 
and the stable locus Deligne-Mumford stable curves without elliptic tails. 

(3) Two weakly pseudostable curves are identified in C\xaw gA // SL 7g _ 7 if and 
only if they have a cusp or an elliptic tail, and their Deligne-Mumford 
stabilizations are the same (see Figure 1). 

As noted in the introduction, the arguments in [9] go through without significant 
change to show that Hilb g 4/ / SL 7g _ 7 ~ M v g '. The fundamental cycle map FC from 
the Hilbert scheme to the Chow variety descends to a map 

vj : mh gA //SL 7g _ 7 ^Chow gA //SL 7g _ 7 . 

By our classification of semistable points, w is an isomorphism away from the locus 
of elliptic tails, is everywhere injective and induces a bijection on closed points. 

The proof of [4, Corollary 6.3] shows that, for any v > 2, the ^-canonical locus 
Hilb ffj „ is smooth at points parameterizing weakly pseudostable curves. In partic- 
ular, Hilb^ 4 is smooth and its good quotient Hilb ff 4 / SL 7g _ 7 is normal. 

Kollar [6, Section 1.6] implies that there is a flat universal family of cycles over 
Chow 9 „ that yields a regular inverse morphism to FC over this locus and his The- 
orem 7.3.1 then shows that Chow gA // SL 7g _ 7 is also normal. Since it is an injective 
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birational projective morphism between normal varieties, vo is an isomorphism. We 
have shown that: 

Theorem 7. Chow gA // SL 7g _ 7 ~ Hilb gA //SL 7g _ 7 ~ M v g . 
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